HRD conformance is a necessary requirement for an encoded video bitstream to be successfully delivered in a practical video streaming system, where the coded bitstream needs to be transmitted to a remote client decoder over networks, and successfully decoded by the decoder for smooth video playout. A decoder has a decoding buffer of a limited size that will incessantly receive the delivered video bitstream from the network. Meanwhile, the decoding module will take out buffered bitstream of each coding unit (e.g. a frame) from the buffer, and decode and playout the reconstructed video signals. In a practical video streaming system, a decoder buffer is necessary so as to absorb the inevitable jittering of channel transmission rate of nowadays networks, and thus, render incessant and smooth decoded video playout.
Therefore, HRD conformance is essentially the same problem as to satisfy the decoder buffer constraint for a coded video bitstream.
By assuming a set of HRD parameters at the encoder and conforming to the assumed HRD parameters in encoding each frame (or generally slice as in H.264), the resultant coded bitstreams are then able to be successfully decoded by any decoder that conforms to the same set of HRD parameters as well. This facilitates the cooperation among encoders and decoders from different manufactures in video codec related industry. The importance of HRD conformance has been widely recognized and standardized in Annex C of the latest H.264/AVC video coding standard.
In general, the concerned HRD or decoder buffer parameters include: the buffer size, the initial buffer level, and the channel rate, which define a commonly known model of leaky bucket. HRD conformance means when filling the coded bitstream data into the buffer and taking out the bitstream for each coding unit decoding, there will be no buffer overflow or underflow. Therefore, to support HRD conformance, a video encoder has to: (i) keep an exact track of the current buffer level after coding each unit (e.g. frame or field); (ii) based on that, derive the upper and/or lower bounds on next unit's coding bits; (ii) make sure that the coded bits of the unit does not go beyond the bounds. In practice, the buffer constraint problem is addressed by the rate control scheme of a video encoder.
Supporting HRD conformance in a GOP-parallel single-pass video encoder with multiple threads presents a challenging problem to address. In this commonly used architecture in real-time video coding systems, multiple GOPs may be encoded at the same time by multiple encoding threads respectively, and each GOP can only be coded once. Although GOP parallel and single pass encoding easily accelerates the encoding process significantly, a side-effect is that supporting HRD conformance becomes a highly more difficult task than that in the conventional case of GOP-sequential single-thread coding. In GOP-sequential video coding, when coding a GOP, all the preceding frames have already been coded, whose coded bits can, thus, be readily used to exactly track the current buffer level and derive correct bounds for HRD conformance coding. However, in the case of GOP-parallel coding, when a thread is going to code the current GOP, some of its preceding GOPs may still be under coding by some other threads. This is illustrated in FIG. 1, where the shadowed areas represent coded part in a GOP. For those GOPs, the exact coded bits are not known when coding the current GOP, which renders HRD conformance coding impossible.
Generally, in a multipass system, one has to let each GOP exactly consume all the budget bits that is allocated to it prior to its actual coding. In that way, for those partially coded GOPs, one can just assume their allocated bits to be their final coded bits, and use them to keep track of the buffer status. However, in single-pass real-time video coding scenarios, exactly achieving the pre-allocated bits for any GOP is a highly challenging task for rate control. Note exact GOP bits achievement is much easier and more straightforward if multi-pass coding is allowed. In that case, even though the pre-allocated bit of a GOP may not be exactly achieved by a coded GOP, one can conduct a HRD conformance check given all the GOPs are coded once. If HRD violations (i.e. buffer overflows or underflows) are found, the encoder can re-encode the trouble GOPs with newly allocated bits to prevent violations. After the 2nd pass coding, the encoder can do the HRD check again, and so on till no HRD violation is resulted. However, this HRD adjustment is impossible in single-pass video encoding, where to support HRD conformance, exactly achieving GOP allocated bits is the only way to go.
The decoder buffer constraint or HRD conformance problem has long been considered an important issue in developing practical rate control schemes for video encoders in video streaming applications. However, so far, most of the existing efforts are focused on the scenarios of GOP-sequential single-thread coding, where satisfying the buffer constraint is generally not a challenging problem in practice.